Claimify-loss ledger: no kept span dies silently (#161) - #166
Merged
Conversation
Every kept Selection span is now accounted for end-to-end: grounding gates emit grounding_rejected with the gate name, and Claimify skips emit claimify_omitted (without double-counting rejected attempts). Bumps the E2 extractor version so the changed decision transcript is a new provenance generation.
…, handler tests Codex review (REQUEST-CHANGES, 3 MAJOR / 3 MINOR) applied: - claimify_omitted attribution is anchored-range overlap ONLY — the text-containment shortcuts let one returned claim suppress omission rows for unrelated keeps sharing text. Orphan rejections (span anchors nowhere or outside every kept range) suppress no omission; unfindable Selection keeps always get their omission row. Deliberate deviation from the review's keep-numbering suggestion: that changes the model-facing Claimify contract (weak-model schema risk, section 2.4); range-only closes the hole code-side. - D56 zero-claim reuse copies the prior chunk's decision transcript forward (new ClaimCatalog.copy_reused_decisions) instead of fabricating a no_info marker over real loss rows; marker only when the prior transcript is empty. - Five handler-level tests drive the real _extract_chunk with canned payloads (zero-return, rejection-only, mixed accept+reject, orphan rejection, unfindable keep); attribution rule break/fail/restore verified. Postgres-gated copy_reused_decisions test added. - Invariant restated as two independent rules (mixed decomposition outcomes); kind and row source_span bounded; D33 (decisions.md) and the postgres schema design amended (enum values, discarded -> ledgered). Gate: ruff clean, pyright 0 errors, pytest 948 passed incl. Postgres-gated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKENhTLJg1HqhbdwCmmkbc
The Compose quickstart workflow hardcodes the post-migration alembic head; the loss-ledger migration moved it. Same pin the migration-graph test already carries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKENhTLJg1HqhbdwCmmkbc
Coverage report
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Claimify-loss ledger: no kept span dies silently (#161)
Closes #161.
Why
On the glm-4.7-flash arm, 207 of 264 kept candidates (78%) produced no claim
and the ledger said nothing about why — Claimify omissions were unrecorded
and the three grounding gates in
_grounded_claimwere silentreturn Nones.Cross-model extraction comparisons could say that a stronger model lands
more claims, never why.
What
claimify_omitted(akept span the model returned no claim for — including Selection spans not
verbatim-findable in the document, which previously vanished traceless)
and
grounding_rejected(a returned claim a D32 gate rejected, with thegate named in
edit_detail:span_not_found/outside_kept_ranges/added_context_unverified, plus the truncated offending text)._grounded_claimreports which gate fired instead of bareNone; theaccept path is behaviorally identical.
claim independently ends accepted or rejected; every keep with no
range-attributable returned claim gets exactly one omission row; orphan
rejections suppress no omission.
(
copy_reused_decisions) instead of fabricating ano_infomarker over areal loss record; the marker is written only when the prior transcript is
itself empty.
ADD VALUE IF NOT EXISTS, additive, no newindexes on the partitioned table);
E2_EXTRACTOR_VERSION→e2-extract-2026.07e:loss-ledger-1(the transcript a replay writeschanged shape).
Review trail
Grok-4.5 implemented the base; Codex (gpt-5.6-sol, xhigh) reviewed with
REQUEST-CHANGES (3 MAJOR / 3 MINOR); this revision applies all findings:
attribution (text-containment shortcuts removed). Deliberate deviation
from the suggested fix (numbering keeps in the Claimify prompt): that
changes the model-facing contract and adds a schema field weak models can
get wrong (§2.4), while range-only attribution closes the same hole
code-side. Orphan rejections and unfindable keeps now behave conservatively
and documented.
no_info— fixed via transcriptcopy-forward with marker fallback, plus a Postgres-gated test.
drive the real
_extract_chunkwith canned Selection/Claimify payloads(zero-return, rejection-only, mixed accept+reject, orphan rejection,
unfindable keep); break→fail→restore verified on the attribution rule.
two independent rules in the design amendment.
kindand the row'ssource_spannow truncate like the rest.updated (enum values, "discarded" → ledgered, loss-row sizing note).
Gate
ruff format/check clean, pyright 0 errors, full pytest incl. Postgres-gated
suites — see checks.